Interactive media system and method

ABSTRACT

Interactive computing systems and methods are provided which enable simple and effective interaction with a user device, which increases interest and improves user experience. The interactive system comprises a user device including a motion sensor, for receiving motion-based gestures through motion of the user device; and a controller, coupled to the motion sensor, configured to control one or more aspects of the system according to the motion-based gestures. Advantageously, the system enables single handed operation of the user device.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a U.S. National Stage patent application ofPCT/AU2016/050285 filed on Apr. 20, 2016, which claims priority to AU2015901421, filed Apr. 20, 2015, and AU 2015904160, filed Oct. 13, 2015.The entire contents of which are incorporated herein by reference.

TECHNICAL FIELD

The present invention relates to interactive digital media. Inparticular, although not exclusively, the invention relates tointeractive display of media and playback of video.

BACKGROUND ART

In the past, almost all media content consumed was professionallyproduced. Radio and television, for example, was professionally producedand broadcast to a large number of users. Now, high quality contentcreation equipment and software tools are within the reach of reasonablyskilled persons, and user generated content (UGC) has become popular.Distribution of such video online via video and social media channels,particularly via mobile devices, is widespread.

A problem with such content is that it is generally not enticing tousers, as it is linear, and generally created for a very wide audience.Furthermore, as more and more content is consumed on portable devices,new challenges are being faced, particularly in relation to navigationin such content.

Typically, traditional desktop media interaction methods have beenapplied to portable devices, which do not provide a good user experienceon these portable devices. For example, touch screen devices oftenincorporate small buttons relating to standard controls, such as pauseand play, which, when shown, typically cover a portion of the video, andare generally inconvenient.

A further problem is that advertising on a mobile device is dominated bytraditional media, and very limited interactivity with the media isprovided. Typically, for a user to respond to a “call to action” inadvertising, he or she must navigate to another website (or app) andmanually take action there. Such action is clearly inconvenient, and asa result, sales opportunities are often lost.

As such, there is a need for improved interactive media systems andmethods.

It will be clearly understood that, if a prior art publication isreferred to herein, this reference does not constitute an admission thatthe publication forms part of the common general knowledge in the art inAustralia or in any other country.

SUMMARY OF INVENTION

The present invention is directed to interactive media systems andmethods, which may at least partially overcome at least one of theabovementioned disadvantages or provide the consumer with a useful orcommercial choice.

With the foregoing in view, the present invention in one form, residesbroadly in an interactive computing system comprising:

a user device including a motion sensor, for receiving motion-basedgestures through motion of the user device; and

a controller, coupled to the motion sensor, configured to control one ormore aspects of the system according to the motion-based gestures.

Advantageously, the interactive media system enables simple andeffective interaction, which increases interest and improves userexperience.

The controller may be configured to selectively control display of mediabased upon the motion-based gestures and a set of media controlinstructions. The media control instructions may define a relationshipbetween the motion-based gestures and the control of the display ofmedia. The media control instructions may be defined in a media controlfile. The media control file may define markers within media files.

The media control file (or the media control instructions) may define adefault video, which is configured to be played when no gestures aredetected, and other video files, wherein each video file is associatedwith one motion-based gesture.

Preferably, the controller is configured to control playback of mediaaccording to the motion-based gestures.

Preferably, the controller is configured to control the playback ofmedia by selecting media for playback. Selecting media for playback maycomprise selecting a media file from a plurality of media files.Alternatively, selecting media for playback may comprise selecting aportion of a media file from a media file. In such case, the portion ofthe media file may be defined by a time on a timeline of the media file,markers within the media file, or by alternative content of the mediafile.

The controller may be configured to selectively control playback ofmedia by receiving a first motion-based gesture and determining a firstplayback configuration in response thereto, and subsequently receiving asecond motion-based gesture and determining a second playbackconfiguration in response thereto

Control of playback of media may be predefined according to themotion-based gesture. Alternatively, the control of playback of mediamay be dynamically defined. In such case, the control of playback ofmedia may be dynamically defined according to when the motion basedgesture was received, or a state of the system at the time the gesturewas received.

Preferably, the media includes visual media. The media may includevideo. The media may include at least one of audio, text, rich text,stylized text, a Hyper Text Markup Language (HTML) document, ananimation, a slide show, a graphic, and an image.

The media may comprise multiple layers of media.

The media may be played back on a display of the user device.Alternatively, the media is played back on a display external to theuser device.

The motion-based gestures may comprise a plurality of predefinedgestures.

The motion-based gestures may be defined by local motion of the userdevice. The motion-based gestures comprise one or a combination of: aleft tilt, a right tilt, a forward tilt, a rear tilt, an inversion tilt,a circular tilt, a flip tilt, a twist, and a shake of the user device.

The system may further include an accelerometer, wherein themotion-based gestures are determined at least in part according to datafrom the accelerometer. The system may include a gyroscope, a compass ora positioning system, wherein the motion-based gestures are determinedat least in part according to data from the gyroscope, the compass orthe positioning system.

The controller may be configured to control presentation of media basedupon the motion-based gestures.

The controller may be configured to control an interactive applicationof the user device. In particular, the controller may be configured tocause the interactive application to purchase an item, add an item to adigital shopping cart or wish list, place a bid on an item, complete atransaction, make a phone call, send an SMS, open an application,communicate with a further device, or call a function of anotherapplication.

The system may further include a media player, wherein the controller isconfigured to control an interactive application of the user devicewhile the media player is playing media.

Control of the interactive application may be independent of theplayback of the media.

The controller may form part of the user device

The system may consist of a handheld device.

The system may include a head mounted display.

The controller may be configured to control the one or more aspects ofthe system according to a set of control instructions and themotion-based gestures, wherein the control instructions define arelationship between the motion-based gestures and the control of theone or more aspects of the system.

In another form, the invention resides broadly in a system forgenerating interactive media, the system comprising a media controlgeneration component, for generating media control data, wherein themedia control data defines a relationship between motion-based gesturesdefined by motion of a user device and control of media, such that auser is able to selectively control the media based upon themotion-based gestures.

The system may further include a camera, for capturing the media in theform of photos or video.

The system may further include a data interface for receiving the media.

The media control generation component may be configured to generateinteractive media based upon motion-based gestures.

In yet another form, the invention resides broadly in an interactivemedia method comprising:

receiving motion-based gestures through movement of a user device; and

controlling presentation of media based upon the motion-based gesture.

In yet another form, the invention resides broadly in an interactivemethod comprising:

receiving motion-based gestures through movement of a user device; and

controlling an interactive application of the user device based upon themotion-based gestures.

Any of the features described herein can be combined in any combinationwith any one or more of the other features described herein within thescope of the invention.

The reference to any prior art in this specification is not, and shouldnot be taken as an acknowledgement or any form of suggestion that theprior art forms part of the common general knowledge.

BRIEF DESCRIPTION OF DRAWINGS

Various embodiments of the invention will be described with reference tothe following drawings, in which:

FIG. 1 illustrates a block diagram of an interactive media displaydevice, according to an embodiment of the present invention;

FIG. 2a illustrates a front view of the interactive media display devicein use in a first configuration;

FIG. 2b illustrates a front view of the interactive media display devicein use in a second configuration;

FIGS. 3a-3f illustrates a plurality of predefined movements of thedevice of FIG. 1;

FIG. 4 illustrates a sequence diagram of an interaction between a userand the device of FIG. 1, according to one embodiment of the invention;

FIG. 5 illustrates a sequence diagram of an interaction between a userand the device of FIG. 1, according to another embodiment of theinvention;

FIG. 6 illustrates a screenshot of a content generation screen of avideo generation system, according to an embodiment of the presentinvention;

FIG. 7 illustrates a block diagram of an interactive media displaysystem, according to an embodiment of the present invention;

FIG. 8 illustrates an interactive media display method, according to anembodiment of the present invention;

FIG. 9 illustrates a diagram of interaction between a user and a device,and a corresponding display of video files on the device, where eachvideo file corresponds to a state of the system where interactions areinterpreted differently, according to an embodiment of the presentinvention;

FIG. 10a illustrates a user device on which video is displayed and inwhich the user may interact through motion based gestures, according toan embodiment of the present invention;

FIG. 10b illustrates the user device of FIG. 10a , on which a favouriteslist is presented;

FIG. 11 illustrates a screenshot of a training screen, where the user istrained to understand the tilt controls of the system, according to anembodiment of the present invention;

FIG. 12a illustrates a screenshot of an interactive shopping screenincluding multi-dimensional navigation, according to an embodiment ofthe present invention; and

FIG. 12b illustrates a further screenshot of the interactive shoppingscreen of FIG. 12a , where the user has navigated in themulti-dimensional navigation hierarchy.

Preferred features, embodiments and variations of the invention may bediscerned from the following Detailed Description which providessufficient information for those skilled in the art to perform theinvention. The Detailed Description is not to be regarded as limitingthe scope of the preceding Summary of the Invention in any way.

DESCRIPTION OF EMBODIMENTS

FIG. 1 illustrates a block diagram of an interactive media displaydevice 100, according to an embodiment of the present invention. Theinteractive media display device 100 enables a user to control displayof media using motion of the device 100, as discussed in further detailbelow. Conveniently, the control of media displayed can be performedwith one hand, as the same hand that holds the device 100 can be used totilt, or otherwise provide motion-based input to the device 100.

The interactive media display device 100 includes a processor 105, adisplay screen 110, coupled to the processor 105, for displaying videoto a user, and a motion sensor 115, coupled to the processor 105, forreceiving motion information of the device 100.

The motion sensor 115 may comprise an accelerometer and a gyroscope,which are used together to determine motion-based inputs. However,alternatively or additionally, the motion-based inputs may be determinedaccording to a compass, a camera, a positioning sensor (such as a GPS orRF positioning techniques) or any other input that can be used todetermine movement of the device 100 such as attitude, rotation rate,acceleration and the like. In a different embodiment, motion basedcommands are generated by observing changes in user positions asidentified by a camera integral to the user device or attached to it.

The interactive media display device 100 includes a memory 120, coupledto the processor 105. The memory 120 includes a media storage component120 c, for storing media for display on the device 100, media decoderinstructions 120 a, for decoding the stored media, and media controlinstructions 120 b, for controlling display of the stored media usingthe media decoder instructions 120 a.

In use, media of the media storage component 120 c is decoded by themedia decoder instructions 120 a, under the control of the media controlinstructions 120 b, and displayed on the display screen 110. Movement ofthe device 100 during display is then monitored for and detected by themedia control instructions 120 b using data from the motion sensor 115.This movement is then used to control display of the media.

The media control instructions 120 b may be configured to controldisplay of a single media file by navigating in the media file. Inparticular, based upon data from the motion sensor 115, the mediacontrol instructions 120 b may cause the decoder instructions 120 a todynamically navigate between points on a timeline of the media file. Asan illustrative example, when the device 100 is tilted to the right(from the perspective of the user), the media control instructions 120 bmay cause the decoder instructions 120 a to jump forward a predefinedamount on a timeline of the media file.

Similarly, the media control instructions 120 b may cause the decoderinstructions 120 a to dynamically switch between alternative content ofthe media file. As an illustrative example, when the device 100 isbriefly shaken, the media control instructions 120 b may cause thedecoder instructions 120 a to switch between alternative media of themedia file, such as different language tracks of the media file.

Alternatively, the media control instructions 120 b may be configured tocontrol playback of multiple media files, by navigating between themedia files, based upon movement of the device 100. As an illustrativeexample, when the device 100 is tilted to the right (from theperspective of the user), the media control instructions 120 b may causethe decoder instructions 120 a to move to a next media file in apre-defined sequence of media files.

According to certain embodiments, the device 100 may be configured tonavigate between various types of media. For example, the device 100 maybe configured to navigate between video and still images throughmotion-based input. This is particularly advantageous if furtherinformation is desired about an item that is shown in a video, and maysuit shopping and advertising use cases.

The media files may be stored locally on the device 100 (for example onmedia storage component 120 c), or downloaded or streamed on demand. Assuch, media may be stored “in the cloud” (i.e. remotely), and updated asdesired. This is particularly advantageous where it is desirable toperiodically update media files, such as advertising, or when short termpromotions are run, without requiring any updates on the device 100.

The device 100 may allow traditional input, e.g. through a touch screenor keyboard, or motion-based input. In some cases, the traditional inputand motion based input is mutually exclusive. As such, traditional inputmay be limited (e.g. a touch screen of the device 100 may bedeactivated) when motion-based input is used. This is particularlyadvantageous where it is likely that a user may accidentally bump thetraditional input when entering motion based input.

The interactive media display device 100 may comprise a smartphone, atablet, or any other suitable portable computing device.

FIG. 2a and FIG. 2b illustrate front views of the interactive mediadisplay device 100 in use. As illustrated, the interactive media displaydevice 100 is generally portable and may be specifically designed to beheld with one or both hands or be placed in a holding frame for easyviewing, and comprises a substantially planar front face 205 includingthe display screen 110.

Initially, a first video 210 a is displayed on the device 100, asillustrated in FIG. 2a . Partway through the playback of the first video210 a, the user tilts the device to the right (from the perspective ofthe user), as illustrated by tilt arrow 215, and as described in furtherdetail below.

In response to detection of the tilting of the device 100 (by the motionsensor 115 and media control instructions 120 b), a second video 210 bis displayed on the device 100, as illustrated with reference to FIG. 2b.

The skilled addressee will readily appreciate that navigation to a thirdvideo (not shown), or back to the first video 210 a, may be providedthrough further tilting of the device 100.

FIGS. 3a-3f illustrates a plurality of predefined movements 300 a-300 fof the device 100. The movements 300 a-300 f are also referred to asgestures, and enable the user to interact with the device 100, asdescribed above.

FIG. 3a illustrates normal playback mode 300 a, where the device 100 isheld upright and with only little movement.

FIG. 3b and FIG. 3c illustrate a left tilt 300 b and a right tilt 300 cof the device 100 respectively. In the case of the left tilt 300 b, atop of the device 100 is tilted to the left, from a perspective of theuser. In the case of the right tilt 300 c, the top of the device 100 istilted to the right, from the perspective of the user.

FIG. 3d and FIG. 3e illustrates a forward tilt 300 d and a back tilt 300e respectively. In the case of the forward tilt 300 d, the top of thedevice 100 is tilted away from the user's eyes, and in the case of theback tilt 300 e, the top of device 100 is tilted towards the user'seyes.

Finally, FIG. 3f illustrates a shake 300 f, in which the user shakes thedevice 100 side to side or up and down or twisting across a vertical ordiagonal axis for a short moment.

Other movements may be defined on the device 100, including a doubletilt (such as left tilt, followed quickly by another left tilt), a longtilt, a slow tilt, a hard shake, a long shake, a twist, a slap, a fastslap and a movement that results in a sudden stop. Furthermore, theabove tilts may be split into a number of sub-tilts, small left tilt(e.g. <30° tilt), a medium left tilt (e.g. >30° and <45° tilt), and alarge left tilt (e.g. >45° tilt). Similarly, a movement may be held(e.g. the device may be tilted and held) or not (e.g. after the deviceis tilted it is returned to its original position).

Furthermore, users may also provide input to the device 100 usingnon-movement based gestures, such as touch screen gestures includingswipe left, right, up or down, pinch or expand, tap, or double tap. Thisis particularly useful if the user is in an environment where movementof the device 100 as input is not suitable or is not convenient for theuser.

According to certain embodiments, the device 100 includes an overviewmode, in which icons showing what gestures are available are presentedto the user. For example, tapping the screen may cause the device 100 toenter or leave the overview mode. This is particularly advantageouswhere different gestures are available according to different settingsof the video(s) as described in the media control instructions. It alsoallows for navigation while the device is laying down flat on a surface,such as a table.

According to certain embodiments, the movements or gestures 300 a-300 fmay be defined in absolute terms. For example, the normal playback mode300 a may be defined as substantially upright with reference to thenormal pull of gravity. Alternatively, the movements or gestures 300a-300 f may be defined in relative terms. For example, the left tilt 300b may be defined by a tilting of the device to the left relative to anearlier position. Relative movements or gestures enable a user to usethe device 100 in various positions, for example while lying down, or inany other circumstance where the device is held in a non-uprightposition during playback.

FIG. 4 illustrates a sequence diagram 400 of an interaction between auser and the device 100, and a corresponding playback of video 410 onthe device, according to a timeline 405. The video 410 comprises aplurality of sequential segments 410 a, numbered 1, 2, 3, etc., and byinteracting with the device 100, the user is able to navigate betweenthe segments 410 a.

Initially, i.e. at time 0 (405 a) on the timeline 405, the user holdsthe device 100 in normal playback mode 300 a. As a result, the video 410plays back normally, i.e. video segments 0, 1, and 2 are played at times0, 1 and 2 respectively.

At time 3 (405 b) on the timeline 405, the user tilts the device 100into a left tilt 300 b. As a result, the device 100 jumps backwards twosegments 410 a of the video 410, and plays video segments 1 and 2 attimes 3 and 4 respectively (rather than video segments 3 and 4 thatotherwise would have been shown). The user may choose this feature if heor she was interrupted and wants to view a portion of the video 410again.

At time 5 (405 c) on the timeline 405, the user tilts the device 100into a right tilt 300 b. As a result, the device 100 jumps forwards twosegments 410 a of the video 410, and plays video segments 5 and 6 attimes 5 and 6 respectively (rather than video segments 3 and 4 thatotherwise would have been shown).

The segments 410 a may be any suitable length, such as 5 or 10 secondslong. Furthermore, the segments 410 a need not be of uniform size. Forexample, each segment 410 a may define a scene of a video, wherein thescene is defined by the content of the video. This information isdescribed in the media control instructions.

The video 410 may comprise a legacy video file, such as a file accordingto the ISO base media file format, the MPEG-4 file format, and mayinclude both audio and video.

FIG. 5 illustrates a sequence diagram 500 of an interaction between auser and the device 100, and a corresponding playback of video files 510a, 510 b, 510 c on the device 100, according to a timeline 505. Thevideos 510 a, 510 b, 510 c comprises a plurality of sequential segments510, similar to the segments 410 a of FIG. 4, and by interacting withthe device 100, the user is able to navigate between the videos 510 a,510 b, 510 c

Initially, i.e. at time 0 (505 a) on the timeline 505, the user holdsthe device 100 in normal playback mode 300 a. As a result, the firstvideo 510 a is initially played.

At time 3 (505 b) on the timeline 505, the user tilts the device 100into a left tilt 300 b. As a result, the device 100 jumps to the secondvideo 510 b, and starts playing the segments 510 from the second video510 b from time 3. The second video 510 b is generally associated withthe left tilt 300 b either implicitly or explicitly.

At time 5 (505 c) on the timeline 505, the user tilts the device 100into a right tilt 300 c. As a result, the device 100 jumps to the thirdvideo 510 c, and starts playing the segments 510 from the third video510 c from time 5. The third video 510 c is generally associated withthe right tilt 300 c either implicitly or explicitly.

The videos 510 a, 510 b, 510 c may be associated with the movements orgestures 300 a-300 f using a media control file. In such case, the mediacontrol file may define which video 510 a, 510 b, 510 c is played uponwhich interaction, i.e. how the device 100 is to respond to the user'smovements and other gestures.

In such case, the media control instructions may be found within a mediacontrol file and may comprise a JavaScript Object Notation (JSON)encoded text file. The media control file and the video files 510 a, 510b, 510 c may be packaged into an archive file and compressed,potentially with a custom file extension to indicate the type ofcontents of the file. The archive file is not only advantageous in thatit is able to compress the files to reduce file size, but also in thatit provides a convenient way of distributing the videos 510 a, 510 b,510 c and the media control file, namely in a single file.

According to certain embodiments, the media control file defines asingle default video, also referred to as a “home clip”, which theplayer plays first, plus up to five other clips, wherein one isassociated with each of the five gestures, left tilt 300 b, right tilt300 c, forward tilt 300 d, back tilt 300 e, and shake 300 f When agesture is made the player plays the associated clip instantly, thenreturns to the home clip when finished.

According to alternative embodiments, the media control file may referto an external location of the video files (e.g. by Uniform resourcelocator (URL)). As such, the video files need not be transmitted withthe media control file, which in turn may reduce bandwidth required toshare content.

To exemplify, the syntax for the media control file may be described asfollows.

{ ″name″: ″Ice″, ″description″: ″Ice drift + fire″, ″created″:″2015-04-27 19:34:23″, ″modified″: ″2015-04-28 12:12:56″, ″start″:″intro″, ″clips″: [ { ″id″:″intro″, ″filename″: ″intro.mp4″, ″end″:″play″, ″play″:″spin-loop″ }, { ″id″:″spin-loop″, ″filename″:″spin-loop.mp4″, ″end″: ″loop″, ″gestures″: {″forward″:″spin-go-forward″, ″back″:″bye″, ″left″:″spin-fix″,″right″:″spin-sitting″, ″shake″:″spin-up-shake″ } }, { ″id″:″bye″,″filename″: ″bye.mp4″, ″end″: ″play″, ″play″:″spin-loop″, ″mute″:true,″vibrate”:true }, { ″id″:″spin-fix″, ″filename″: ″spin-fix.mp4″, ″end″:″play″, ″play″:″spin-loop″ }, { ″id″:″spin-sitting″, ″filename″:″spin-sitting.mp4″, ″end″: ″play″, ″play″:″spin-loop″ }, {″id″:″spin-up-shake″, ″filename″: ″spin-up-shake.mp4″, ″end″: ″play″,″play″:″spin-loop″ }, { ″id″:″spin-go-forward″, ″filename″:″spin-go-forward.mp4″, ″end″: ″play″, ″play″:″quad-loop″ }, {″id″:″quad-loop″, ″filename″: ″quad-loop.mp4″, ″end″: ″loop″,″gestures″: { ″forward″:″quad-overtake-forward″,″back″:″quad-rear-back″, ″left″:″view-high″, ″right″:″view-boom″,″shake″:″moto-loop″ } }, { ″id″:″quad-overtake-forward″, ″filename″:″quad-overtake-forward.mp4″, ″end″: ″play″, ″play″:″quad-loop″, }, {″id″:″quad-rear-back″, ″filename″: ″quad-rear-back.mp4″, ″end″: ″play″,″play″:″quad-loop″ }, { ″id″:″view-high″, ″filename″: ″view-high.mp4″,″end″: ″play″, ″play″:″quad-loop″ }, { ″id″:″view-boom″, ″filename″:″view-boom.mp4″, ″end″: ″play″, ″play″:″quad-loop″ }, {″id″:″moto-loop″, ″filename″: ″moto-loop.mp4″, ″end″: ″loop″,″gestures″: { ″forward″:″outro″, ″back″:″moto-rear″, ″left″:″moto-left″,″right″:″moto-right″, ″shake″:″quad-loop″ } }, { ″id″:″moto-rear″,″filename″: ″moto-rear.mp4″, ″end″: ″play″, ″play″:″moto-loop″ }, {″id″:″moto-left″, ″filename″: ″moto-left.mp4″, ″end″: ″play″,″play″:″moto-loop″ }, { ″id″:″moto-right″, ″filename″: ″moto-right.mp4″,″end″: ″play″, ″play″:″moto-loop″ }, { ″id″:″outro″, ″filename″:″outro.mp4″, ″end″: ″finish″ } ] }

In particular, an “id” field may be used to identify each video clip, a“filename” field may be used to associate a filename with the videoclip, an “end” field may define what is to happen when an end of theclip is reached, and a “gestures” structure, together with gesturefields such as “forward”, “back”, “left”, “right” and “shake”, whichdefine which video clip is to be played upon detection of thecorresponding gestures.

In addition to the above fields, the media control file may furtherinclude metadata relating to the media, such as where the media wasfilmed or where it relates to. This could then allow other users to findmedia close to their location, or search for media relating to a certainlocation or use map searching features to find the media.

Content may be professionally made for the device 100, or usergenerated, as outlined below.

FIG. 6 illustrates a screenshot 600 of a content generation screen of avideo generation system, according to an embodiment of the presentinvention. The video recording system may comprise a smartphone, apersonal computer or the like.

The content generation screen includes a plurality of gesture buttons605, which correspond to the gestures, left tilt 300 b, right tilt 300c, forward tilt 300 d, back tilt 300 e, and shake 300 f, and static. Thegesture buttons enable the user to select the gesture to determine whathappens at the decoder on detection of said gesture.

The content generation screen includes a record button 610, whichenables the user to capture video in relation to the selected gesture.An alternative button may be provided to enable the user to select apre-recorded video, from storage local to the system, from a remotesystem, or from a further device. This is particularly useful if acamera is not available on the system.

In use, the user navigates between the different gesture buttons 605,associates a video with some or each of the associated gestures, andsaves the file. The system then automatically generates the mediacontrol instructions, and stores it in an archive together with thevideo clips to simplify the sharing of the generated content.

In other embodiments, an advanced content generation screen may beprovided. The advanced content generation screen may enable any numberof media clips or files to be linked to motion gestures. As anillustrative example, the advanced content generation screen may definehow motion gestures are interpreted for each clip or file. As a result,complex navigation can be provided where, for example, certain videoclips can be navigated to only via certain other video clips.

According to certain embodiments, the video generation system includesan accelerometer, gyroscope, compass and/or other motion sensor, whichit monitors while recording. Upon detection of a gesture, the system mayautomatically select the associated gesture button 605, continue torecord, and associate the continued recording with the gesture.

According to certain embodiments, the video generation systemautomatically splices and trims video clips, and/or determines how theyrelate to other clips, based on how the user was holding the devicewhile recording.

The content generation screen may include the ability for a user tocapture an image, and use the captured image to assist in aligningsubsequent videos. In such case, the captured image may be overlaid overthe captured video, to enable the user to align an item in the scene ascaptured (e.g. a person) to an item in the captured image (e.g. an imageof the person). As a result, the user can ensure that the item beingrecorded is at approximately the same place at important points of thevideo, such as start and end points, to provide alignment.

According to certain embodiments, the video generation system enableshands free recording by automatically recording clips one after theother, and automatically associating the recorded clips with gestures.The user may pre-configure the sequence of clips, or use the defaultconfigured clip sequence, and an audio or visual indicator may beprovided to the user to let the user know when a video recording isabout to start or finish and may additionally indicate which gesturethey will correspond with.

According to certain embodiments, the media may be manipulated toproduce more desirable transitions between media. As an illustrativeexample, items in the media may be aligned across a transition, visualeffects may be added, such as cross-fade effects, fade through blackeffects, and the like, to various parts of the media, such as at thebeginning and the end of a video.

Additionally, the media may be manually manipulated to produce a moredesirable effect for when users are later consuming that media and arenavigating between the different media. Such functions that may helpfacilitate the alignment of items between media may include zooming,panning, cropping, etc.

FIG. 7 illustrates a block diagram of an interactive media displaysystem 700, according to an embodiment of the present invention. Theinteractive media display system 700 is similar to the interactive mediadisplay device 100 of FIG. 1.

The interactive media display system 700 includes a portable computingdevice 705, in the form of a smartphone, a media streaming server 710,and a media control server 715.

In use, video data is streamed from the media streaming server 710 tothe portable computing device 705 by a data network 720, such as theInternet. As the portable computing device 705 is moved, motion of thedevice 705 is detecting using, for example, an accelerometer andgyroscope, and data relating to the motion provided to the media controlserver 715.

The media control server 715, upon detection of a valid motion gesture,as described above, will instruct the media streaming server 710 toeither change the video file, or move to a different part of the videofile. New video data is then streamed from the media streaming server710 to the portable computing device 705.

Alternatively, the media control server 715 may provide controlinstructions to the portable computing device 705, which is then used bythe portable computing device 705 to instruct the media streaming server710. In such case, the media control server 715 may, for example,comprise a Hypertext Transfer Protocol (HTTP) server. Additionally, thestreaming server may provide live content.

Advantageously, only the relevant data is streamed to the portablecomputing device, which decreases bandwidth usage. This is particularlyuseful when there are a large number of possible interactions, each withan associated video, as only the videos to be played back need to beprovided to the user.

According to certain embodiments (not illustrated), a media playerincorporating motion based video control, is provided in a web browserof a user device. Advantageously, the media player is configured to usemotion data, if available, and provide alternative navigation elements(e.g. buttons, or voice commands corresponding to the gestures) if notavailable.

For example, non-motion based data input, such as keyboard or mouseinput, may be translated into motion gestures to enable playback ofmedia according to the present invention. In such case, a left arrow keymay be translated into a left tilt, a right arrow key may be translatedinto a right arrow tilt, and so on.

According to certain embodiments (not illustrated), an application isprovided with motion-based interaction, wherein the media player isembedded using a third party or native API. As an illustrative example,the media player may be provided in a WebView of an application usingthe WebView class of the Android API provided by Google Inc. and theOpen Handset Alliance, or using the UIWebView class of the iOS DeveloperLibrary provided by Apple Inc. Alternatively, the media player may beprovided by a native library of the device.

In such case, a relationship between the content and the interaction maybe defined in content provided to the media player, such as throughJavaScript code in a HTML page. In such case, the HTML page may definethe media to be played back, as well as interactions or operations ortasks to be performed upon detection of motion-based commands by thedevice.

An example use case of such motion based control is in video basedadvertising on a social media system such as Facebook news feed. In suchcase, a video may be played back in which the user may register theirinterest in a product, like a page, purchase a product, or navigate toan external page through motion-based commands. The video may include anaudio or text-based instruction (e.g. “Shake your phone to like ourpage”), or such commands and actions may be predefined (e.g. a tiltbackwards and forwards may correspond to yes, a tilt sideways maycorrespond to a no).

In another example, the user may be presented with various images (orother media) in which they are prompted to provide input on, such as to‘like’ or ‘dislike’ the image. For example, the user may tilt left ifthey don't like the image, or tilt right if they do like the image.Alternatively, the user may shake the device to “pass” on an offerassociated with the image. The user is then presented with a next image(or other media) for further input, which may be random, in a list, orcustomised according to earlier input. Such embodiments are particularlysuitable for dating apps or websites, where it is desirable toefficiently select among users.

The motion-based commands may be translated to an action through a URI.For example, upon shaking the phone, a URI may be called in which theuser may “like” a page, and upon tilting the phone, a URI may be calledwhere the user may purchase a product.

In certain cases, the URI may be used to make a background HTTP requestto a server, to avoid disrupting the media which is being played back.As an illustrative example, a user may be watching a fashion video,where the user is able to register their interest in, purchase, like orvote for a fashion item during the fashion video, and without disruptingthe video.

According to certain embodiments, an application is provided in whichrecording of media is provided with motion-based control of therecording. In such case, the recording functionality may be providedthrough a web browser, a WebView (discussed above) or a native filelibrary within another application.

According to certain embodiments, in addition to be able to controlplayback of video based upon motion of a device, the user creating themedia is able to select and control other aspects of the device, such asa background soundtrack to be played to the end user, muting audio of avideo clip, activating a vibrate function of the device.

According to some embodiments, the system 700 includes a social mediacomponent. The social media component enables chat between users, publicand private comments relation to content, and a rating system.Furthermore, the social media component enables users to share content,for example by setting a status of the content to public, or storeprivate content.

According to certain embodiments, sharing of content is achieved bymoving (e.g. tilting) the device. Navigating to and selecting content toplay can also be achieved by moving the device. Furthermore, a shakemovement at certain times may cause random media to load and startplaying, and thus a movement need not be linked to media in apre-defined manner. In some systems, all interaction with the system maybe provided through motion of a device.

According to certain embodiments, content is provided for limitedviewing. In particular, content may be configured to be played a certainnumber of times, within a specified time period and/or on specificdevices only. The content might be configured to report usageinformation, like play frequency, date, time, device back to the contentcreator. Deletion of the content could be triggered on certain events orthe content creator could decide to remotely delete the content.

According to certain embodiments, an external motion sensor may be usedto provide input to, and control a device. For example, a portabledevice such as a smartphone or tablet may be linked with a furtherdevice that includes a display, such as a desktop/laptop computer or aTV or projector, wherein a motion sensor within the portable device maybe used to detect movement thereof, to control display of content on thelinked further device. In short, motion of the portable device may beused to control media on the further device.

The connection between the portable device and the further devices maybe established by any suitable means including by: scanning a QR codedisplayed on the screen of one device with the camera of the other, andusing the code to link the devices; entering a unique code displayed onthe screen of one device into the software on the other; navigating to aunique URL on one device that is displayed on the other; being loggedinto an account (eg. by a username and password) on both devices at thesame time; sending an SMS message to the portable device including anURL or other code to link the devices; using the GPS location of thedevices to connect devices in proximity to one another; interacting witheach device simultaneously, such as by tapping a button on each deviceat (about) the same time; playing a tone or series of tones on thespeakers of one device such that the microphone of the other device candetect the tone to establish the link; or by using an RFID or Bluetoothlink between the devices.

In one example of the above embodiment once a suitably persistentconnection has been established with a further device, software on theportable device detects a motion-based gesture of a plurality ofpredefined movement gestures (eg. right tilt, left tilt, etc.), andtransmits details of that gesture, along with suitable identificationinformation, such as a unique code, to a server. The server thencommunicates that information to the further device which may act as amedia player. The further device receiving the information then changesthe display of media in response the information received. Furthermore,other functions on the further device can be initiated based upon thereceived gesture, and such functions can be hardware or softwarerelated.

The controlling portable device may also display the same interactivemedia on its screen, or display other information, such as what movementwas just recognised, or display buttons to disconnect, pause the media,and provide alternative tappable navigation buttons.

There may be a plurality of further devices configured with a suitablypersistent connection with the portable device. Multiple users maycontrol different further devices or peripherals. Furthermore, users maybe spatially separated, for example, and multiple users, anywhere in theworld, may simultaneously watch or control any other of the furtherdevices.

In particular, one user's portable device could control the media suchthat one or many other people in any location around the world couldexperience the same control of media, based on the commands received.

Furthermore, the system described above may be used to control a headmounted display for Virtual Reality purposes. This allows forinteractive Virtual Reality experiences for a user in different physicalorientations, for example when they are stationary, lying down, sittingin a non-swivel chair and keeping their head still. Voice commands mightalso be used to control aspects of the Virtual Reality interactive mediaexperiences, as detected by either device.

In the case of multiple users using the system, the control of the mediamay be passed between users, either manually, or automatically, such asafter a period of time.

It is also possible to for multiple people to simultaneously control theexperience that multiple people are watching.

Additionally, an audio link could be maintained to allow users to talkwith each other during display of the media.

As discussed above, different videos may be associated with each of theavailable gestures. However, according to further embodiments of theinvention, more complex interaction may be used to navigate between alarge number of videos. For example, a gesture may be interpreted in thecontext of a previous gesture. As such, complex branches of interactionmay be defined, enabling a truly customised experience to the user.

FIG. 8 illustrates an interactive media display method, according to anembodiment of the present invention.

At step 805, a default video is played to the user. The default videomay be defined by a media control file, as discussed above, and isgenerally played automatically and may be looped.

At step 810, a first motion gesture is received from the user. The firstmotion gesture relates to a motion of a portable device, such as a tilt,a shake or the like.

In response to the first motion gesture, a first video is played. Inparticular, the first motion gesture is associated with the first videoin a media control file, as discussed above.

Upon completion of the first video, the default video is then playedagain in step 820.

At step 825, a second motion gesture is received from the user. Thesecond motion gesture relates to another motion of a portable device,such as a tilt, a shake or the like.

In response to the second motion gesture, a second video is played. Inparticular, the second motion gesture is associated with the secondvideo in a media control file, as discussed above.

According to certain embodiments, control of the media is only providedin certain scenarios and not others. For example, user may be forced towatch certain video clips, such as advertisements. In some embodiments,the system is configured to respond to motion during a home (or default)video, while not responding during other videos.

According to certain embodiments, the motion-based media controlinstructions may be stored in a backwards compatible metadata portion ofa video file. As such, a legacy video player may be able to play themedia file without any motion based control, whereas a more advancedplayer may provide detailed motion based interaction with the same file.Alternatively, the motion-based media control instructions may be storedin a HTML file in which the video file is referenced, for example in theform of JavaScript code.

The metadata portion may also include other information about the videofile, such as chapter information, which may be used together with themotion-based media control information to provide more intelligentnavigation in the video. Examples of such navigation include go to end,pause, slow motion, loop, define “chapters” to jump to, play only funnysections, skip unnecessary parts, speed up certain parts and the like.

FIG. 9 illustrates a diagram 900 of interaction between a user and adevice, and a corresponding display of video files on the device,according to an embodiment of the present invention. In particular, eachvideo file corresponds to a state of the system where interactions areinterpreted differently.

Initially, a first video 905 is played to the user. Upon detection of aleft tilt 300 b, a second video 910 is played to the user, after whichthe first video 905 is again played. Similarly, upon detection of aright tilt 300 c, a shake 300 f, a forward tilt 300 d, or rear tilt 300e, a third video 915, a fourth video 920, fifth video 925 or sixth video930 is played respectively.

Upon receipt of a shake 300 f during playback of the third video 915, aseventh video 935 is played. Similarly, upon receipt of a right tilt 300c during playback of the seventh video 935, an eighth video 940 isplayed.

During playback of the eighth video 940, several motion-based commandsmay be interpreted. In particular, upon receipt of a forward tilt 300 dor a back tilt 300 e, a ninth video 945 or tenth video 950 is playedrespectively. Additionally, upon receipt of a shake 300 f duringplayback of the eighth video 940, the first video 905 is again playedand the process may be repeated.

Finally, upon receipt of a right tilt 300 c during playback of the ninthvideo 945, an eleventh video 955 is played.

Upon completion of playback of any of the second to eleventh videos910-955 without any motion input, the first video 905 may again beplayed. Alternatively, and as discussed below, playback may cease uponcompletion of a particular video and a website may instead be displayed.

As illustrated above, the right tilt 300 c is interpreted differentlydepending on what video is being played. For example, during the firstvideo 905, the right tilt 300 c results in playback of the third video915, whereas during playback of the third video, the right tilt 300 cresults in playback of the seventh video 935. In short, eachmotion-based input may be interpreted in light of the video beingplayed, which enables unlimited interaction with the user.

According to some embodiments, a website is automatically opened uponcompletion of one or more of the videos. For example, upon completion ofa video or a set of videos 905-955, a website may be provided thatenables purchase of further media. The website may be opened byrequesting a URL on the device in which the media is played, the defaultweb browser, or by passing an identifier to another app or website, suchas an online auction app or website. The website may open automaticallyupon the recognition of a motion-based command, after the video clipfinishes, or upon reception of another gesture or tap button/s orelement/s on the screen. Additionally, a HTTP request may be made to aserver including instructions to perform any type of transaction withthe server. This may happen without any change of the media, while avideo is playing and without any indication to the user.

As a result, the system may facilitate the purchase of anything directlyor indirectly related to the content (videos) generally, or to aspecific video 905-955. In practice, the process could be used to obtaininformation from the user, for example to select a product, size, style,colour, etc. then facilitate the purchase or discovery of moreinformation about the product.

The above example describes how embodiments of the present inventioncould facilitate a purchase of a product or many products without theuser touching the screen. Items could be added to a wish list orshopping cart with the checkout and payment process happening at a latertime, all using motion-based commands. Items in the shopping cart orwish list could persist across many uses of the application, by savingthe associated data to local or remote storage or both.

Further, the payment process may be initiated by a single motion-basedcommand. Payment details may have been collected at an earlier time.This effectively allows for a one-motion purchase, such as a one-tiltpurchase, a one-slap purchase, a one-twist purchase, etc.

Additionally the user's size, colour preferences, style preferences,etc. could persist across many uses of the application and influencewhat items are displayed to the user, according to their savedpreferences. As an illustrative example, only clothes that would belikely to fit the user may be shown. The system may additionally onlyshow items that are currently in stock in the user's size. Thesepreferences could be explicitly set by the user however may also becalculated by the system, for example the system could take intoconsideration user browsing habits, previous purchases, machine learningtechnologies, or any other method that may allow the system toselectively prioritise displaying certain items instead of other itemsto the user for whatever reason, possibly for the purpose of increasingsales in relation to the prioritised items.

As a further example, a user may be shown different media based on theiruser profile. For example, a male user may be shown male clothing ormale related items, likewise a female user may be shown female clothingor female related items.

Further to saving the user's preferences as described in the aboveparagraphs, the system may additionally allow the user to provide one ofmore images to be displayed together with images or videos of items.This is particularly useful to help the user decide on the suitabilityof an item. The provided images or parts of the images may appearsuperimposed over the items (or vice versa) to simulate the item in use,however the images may appear in other places in relation to the item,such as next to the item, under the item, etc.

The system may allow the user to take a photograph using the camera onthe device, select an image from local storage, provide an image from aremote source, or any combination of these.

Preferably the image includes a person and software may automaticallycrop out or select the head of the person. Additionally or instead ofthis automatically happening, software may provide features allowing forthe user to perform a crop or selection operation manually. In practice,when the cropped image of a person's head is displayed with an item suchas an item of clothing, it may be superimposed over the item where thehead should appear if the person was wearing that item. This isadvantageous, as it should allow the user to more easily decide if anitem of clothing is suitable for that person. This feature is, however,not limited to items of clothing, and could include any type of item,such as cars, furniture, handbags, wristwatches, etc.

According to certain embodiments (not illustrated), the system mayfacilitate an auction where many users can bid for items usingmotion-based input. The movement of the device could trigger bidding anda multi-dimensional navigation menu could support bidding on multipleitems almost simultaneously. In other embodiments the system mayfacilitate gambling related functions through movement of the device,for example selecting race or competitor and expediently placing a bet.

According to certain embodiments (not illustrated), movement of thedevice can be used to trigger functionality of the device, such asmaking phone calls and sending SMS; opening apps, calling functions ofthe app that the player is running within, such as calling Like andSharing functions in a social media application; calling functions ofother apps and passing data to other apps, such as a URI.

It should be noted that it is advantageous to the user to be able toview media in full-screen mode and still be able to interact with themedia (or the device generally). As such, there is no need to waste partof the screen on graphical elements, such as buttons, or waiting for avideo to end to be able to interact with the device. Furthermore, it maybe more convenient to interact with the device using motion-basedcommands, rather than, for example, tap the screen or click a button toinitiate an action. For instance, tapping the screen will cause theuser's finger to cover part of the screen momentarily, blocking thescreen from the user's view.

According to certain embodiments, one or more of the videos 905-955 maybe buffered or preloaded either from local files, remote files orstreaming servers during playback of another video 905-955. This may beperformed in anticipation of commands of the user.

In some embodiments, the device may buffer a first portion (e.g. 10seconds) of all video files to which the user may directly navigate fromthe current video file. In the case of the diagram 900, during playbackof the first video 905, the device may buffer a first portion of each ofthe second, third, fourth, fifth and sixth videos 910-930. In otherembodiments, the video files to which the user may directly navigatefrom the current video file may be downloaded in their entirety, ratherthan just a first portion thereof.

As a result of the buffering (or file download), the device is able todirectly play the requested content upon recognition of the gesture,which reduces lag and improves user experience.

In a further embodiment of the invention, an interactive application(e.g. a shopping application) is controlled according to motion-basedgestures, while video (such as an advertisement) is being displayed onthe user device, as outlined below.

FIG. 10a illustrates a portable user device 1005 in a playbackconfiguration according to an embodiment of the present invention. Inparticular a video 1010 is displayed, in full screen, on the portableuser device 1005, while a controller (not illustrated) is configured toplace certain items in a favourites list according to motion-basedgestures.

In particular, the video 1010 is associated with media controlinstructions, which define actions according to time points in the video1010. As such, different items may be placed in the favourites listaccording to when the user interacts with the video 1010.

In the present example, tilting the device 1005 left or right isassociated with moving the video to back and forward between timepoints, and tilting the device forward is associated with adding an itemto a favourites list, dependent on the current time of the video (e.g.between two time points). However, the creator of the content may createany suitable association between gestures and actions, and suchassociation may change as the video is played back.

The video 1010 may display items 1015, such as clothing, gadgets or thelike, which are available for purchase. In such case, several differentitems 1015 are typically displayed throughout the video, and often in acontext. For example, in the case of clothing, the clothing may beillustrated in use (i.e. worn), and similarly in the case of gadgets,the gadgets may be illustrated being used.

According to certain embodiments, training instructions are renderedinto the video 1010 at the beginning of the video. In particular, thetraining instructions illustrate what action (e.g. fast forward, add tofavourites) each gesture (e.g. tilt left, tilt forward) is associatedwith. This is particularly important when the video 1010 is displayedfull screen, as no room is left on the display of the user device 1005for such instructions.

When the video is finished, the user sees a list of items they selectedusing the tilting gestures, as shown in FIG. 10b . In particular, theportable user device 1005 displays a favourites list 1020, comprising aplurality of shopping elements 1025. The user can then go on to purchasean item, or find out more about the item, by tapping on thecorresponding shopping element 1025.

This embodiment thus illustrates one example of user interaction with aportable device while a full screen video (or other media) continues toplay or be displayed. Furthermore, that full screen video may be playedfrom within another application, for example a social media applicationsuch as Facebook.

As can be seen from the favourites list 1020, multiple items can beadded to the list depending on when the user interacted with the device1005. This is particularly useful when multiple items are displayed inthe video, each of which is available for purchase.

The skilled addressee will readily appreciate that any suitable actionmay take place upon receipt of a motion-based gesture. For example,items may be added to a shopping cart, and/or actual purchases couldoccur while video is playing.

As discussed above, it is often advantageous to provide training to theuser prior to playing media. FIG. 11 illustrates a screenshot of atraining screen, according to an embodiment of the present invention.

The training screen includes a plurality of gesture elements 1105, and aplurality of action elements 1110, wherein each action element 1110 isassociated with a gesture element. In the example illustrated in FIG.11, a tilt to the left moves back in the video and a forward tiltresults in a purchase (buy) of the currently presented item.

The gesture elements 1105 are advantageously animations, illustrating amotion-based gesture with reference to the user device. For example, thegesture element 1105 relating to the right tilt may be configured toillustrate an animation of a right tilt.

Furthermore, the training screen may be configured to detect gestures,and highlight an associated gesture element 1105 upon receipt of thegesture. For example, the gesture element 1105 relating to the righttilt may be highlighted upon receipt of a right tilt gesture, to confirmwith the user which gesture is received. This is particularly useful asthe user practices motion based gestures.

After viewing the training screen, the user is able to tap a start videobutton 1115 to start the video.

The media control instructions may define a multi-dimensional navigationhierarchy. For example, the user may navigate back and forward inside avideo, as well as in and out of the video, for example to websites orother videos. As an illustrative example, certain gestures may be usedto navigate back and forward in a video, whereas another gesture maycause a web page to be displayed. In addition, different web pages maybe displayed based according to where the user is on a timeline of thevideo.

FIG. 12a illustrates a screenshot 1200a of an interactive shoppingscreen, according to an embodiment of the present invention. Theinteractive shopping screen provides a 4-way navigation hierarchy, asdiscussed above

As shown in FIG. 12a , the interactive shopping screen comprises animage 1205 of an item, and an associated “add to cart” button 1210.Furthermore, first and second navigation elements 1215 a, 1215 b aredisplayed to provide an indication to the user of where they are in themulti-dimensional navigation hierarchy.

In particular, the first and second navigation elements 1215 a, 1215 beach define possible positions 1220 and an actual position 1225 in arespective dimension. FIG. 12a illustrates the leftmost position in aleft/right direction as the actual position (see the first navigationelement 1215 a) and the rearmost position front/rear direction as theactual position (see the second navigation element 1215 b).

As the user navigates in the 4-way navigation hierarchy, the image 1205may be updated to illustrate alterative configurations of the same item,or to display alternative items.

FIG. 12b illustrates a further screenshot of the interactive shoppingscreen, where the user has navigated forward four steps and right threesteps in the 4-way navigation hierarchy. The further screenshot includesa playback bar 1230, which enables the user to control playback of avideo. In particular, the image 1205 may be replaced by a video of theitem.

When the user wishes to purchase one of the items, he or she may do sousing the “Add to cart” button 1210. Alternatively or additionally, theuser may purchase items using motion based gestures, as discussed above.

In some embodiments, the system may be configured to automatically, orsemi-automatically generate gesture-based control data, according to oneor more websites. In such case, the user may enter a URL or multipleURLs of websites, from which the system extracts information such asmedia, product description, prices, etc. This is particularly useful inconverting traditional web-base stores into motion gesture-basedinteractive shopping systems.

In the example of FIGS. 12a and 12b , the first navigation element 1215a may be associated with content from different URLs, and the secondnavigation element 1215 b may relate to different content from a singleURL.

The generated media control instructions may reference the mediadirectly from the source (i.e. the original URLs), or the media may becopied to another server (such as a content delivery network) forserving the media. The interactive advert may direct the user to theentered URL when a purchase is desired, or purchasing may happendirectly as described earlier.

In yet alternative embodiments, one or more product feeds may be usedinstead of URLs to generate the gesture-based control data. In suchcase, as data is received on the product feed, the information, such asmedia (or location of media), product description, prices, may beautomatically extracted therefrom and used in a similar manner to thatdescribed above to generate the control data.

The system may be configured to continuously update the control data, orgenerate the control data once off (e.g. in response to receiving aninstruction from the user). For example, in some embodiments theproducts displayed may change according to the information in theproduct feed at that time.

According to certain embodiments, the system may be further configuredto receive preferences from the user, such as styles of GUI elements,colours, number of products to display, and generate the control databased thereon. In this context, the user may be setting thesepreferences for themselves or for other users who will later use thesystem.

According to some embodiments, the display device can be held in eithera portrait or landscape orientation by the user. In such case, the mediais played back in either portrait or landscape mode, such that it isoriented correctly during playback, and motion-based input is detectedaccording to the orientation the device is being held.

According to certain embodiments, the player may collect statistics andusage information. The player may then send this information to a serverfor analysis. Such data may allow for understanding a user's interests,creating a user profile, targeting and retargeting of advertisements,changing the nature of the content based on the users' profile, etc.

The above mentioned data collection process could additionally allow fora self-learning piece of technology, which could identify trends andother learnings to apply back into the system.

According to certain embodiments, the present invention may includeDigital Rights Management (DRM). The DRM may be applied to each of thevideos 905-955, and to any media control file(s), either as a group orindividually, to prevent the content from being re-distributed in anunauthorised manner.

According to some embodiments, the motion-based interaction with thesystem is context aware. In particular, one or more of the motion-basedinteractions 300 b-300 e may be interpreted differently depending on thecontext, or different videos may be played depending on the context.

Examples of contexts in which the system may adapt include time of day,weather, time of year, season, current location, and relative location.As an illustrative example, if viewing at night, different clips mayplay compared with viewing during the day; and if viewing when theseason is winter, a different clip might play compared with viewingduring summer.

The above examples focus primarily on the use of an accelerometer and/orgyroscope to determine motion-based inputs. However, alternatively oradditionally, the motion-based inputs may be determined according to acompass, a camera, a position sensor (such as a GPS) or any other inputthat can be used to determine an attitude, rotation rate, accelerationor other aspect of a device. In a different embodiment, motion basedcommands are generated by observing changes in user positions asidentified by a camera integral to the user device or attached to it.

In some cases, the device is coupled to the user as a head mounteddisplay. In such case, commands may be generated by the user tiltingtheir head left or right, or forwards or backwards, for example, toselectively control display of the media.

Most portable devices have a camera which can be used to track headmovements of the user and trigger an associated action based on thatmovement, for example left head tilt triggers the left gesture action, aright head tilt triggers the right gesture action, etc. This is,however, not limited to head movements. Hand movements, body movementand movement of objects as seen by the camera could also be used. Thiswould be very useful if someone had limited use of his or her hands. Inthe present invention the movement of the user replaces the movement ofthe device.

According to certain embodiments, the motion-based inputs are determinedacross three axes. The motion based input may relate to movement alongthe axes, rotation around the axes, or a combination thereof.

Embodiments of the present invention may be used in a variety ofdifferent scenarios, including for entertainment (e.g. funny interactivevideos, interactive adventures), training (e.g. tutorials where a useris able to easily replay sections of the video as required), education(e.g. where a user is able to answer a video question and thus navigateto the next question by making the correct gesture), virtual tours,games, online shopping, dating websites, or when single handed operationof the device is desirable.

In the case of a single video, motion may be used to initiate fastforward, slow forward, jump forward, fast rewind, slow rewind, jumpback, pause and similar functions, in a similar manner to a joystick orother user input device. This may be particularly useful when viewingsecurity camera footage on a mobile device.

A further example of the present invention relates to Virtual Reality.When the device is mounted to a user's head by a suitable mountingdevice, the control of the media and calling of functions can be madevia head movements. In this example, the system may display the media ina suitable way for Virtual Reality experiences, such as in split-screenview. In such case, a first type of movement, such as a panning movementof the device, may be used to influence the media by a virtual realitymodule (e.g. provide Virtual Reality by tracking movement of the user'shead), and a second type of movement, such as an abrupt noddingmovement, may be used to change content of the media (e.g. changescenes) or otherwise influence the media.

The system may then allow for both recording and playing of such VirtualReality experiences. The manipulation of the media so that it is moresuitable to Virtual Reality experiences may happen while recording themedia, after recording the media and before playing, whilst the media isplaying, or any combination of these. The manipulation may take place onthe device, on a further device or on a remote server and anycombination of these.

While the media control instructions have been primarily described withreference to a relationship between a motion-based gesture and media tobe displayed, the media control instructions may also control thedisplay of media in the absence of motion-based gestures.

For example, the media control instructions may cause a section of video(e.g. between 2 markers) to loop infinitely (or a certain number oftimes), to pause the media and wait for additional user input, to pausethe media when a first gesture is detected and continue when anothergesture is detected, or the like.

While the above interaction has been described with reference tomotion-based interaction, where motion alone is used to identify thegesture, in some embodiments, the user may touch the screen while movingthe device to provide a gesture.

In some cases, gestures may be interpreted differently, or be associatedwith a different function, depending on whether the user is touching thescreen or not while moving the device. Alternatively, gestures may onlybe received (detected) while the user is touching the screen to preventinadvertent detection of gestures based upon accidental movement of thedevice.

In some embodiments, the user may touch anywhere on the screen whilemoving the device. Alternatively, a graphical element, such as a button,may be used, either to activate gesture recognition, or to modifygesture recognition.

According to certain embodiments, the system may be configured to turnon and off gesture recognition using a particular gesture. For example,the user may turn on gesture recognition by shaking the device, andlater turn off gesture recognition by again shaking the device. This isparticularly advantageous in reducing accidental gesture control ofmedia.

In the present specification and claims (if any), the word ‘comprising’and its derivatives including ‘comprises’ and ‘comprise’ include each ofthe stated integers but does not exclude the inclusion of one or morefurther integers.

Reference throughout this specification to ‘one embodiment’ or ‘anembodiment’ means that a particular feature, structure, orcharacteristic described in connection with the embodiment is includedin at least one embodiment of the present invention. Thus, theappearance of the phrases ‘in one embodiment’ or ‘in an embodiment’ invarious places throughout this specification are not necessarily allreferring to the same embodiment. Furthermore, the particular features,structures, or characteristics may be combined in any suitable manner inone or more combinations.

In compliance with the statute, the invention has been described inlanguage more or less specific to structural or methodical features. Itis to be understood that the invention is not limited to specificfeatures shown or described since the means herein described comprisespreferred forms of putting the invention into effect. The invention is,therefore, claimed in any of its forms or modifications within theproper scope of the appended claims (if any) appropriately interpretedby those skilled in the art.

1. An interactive computing system comprising: a user device including amotion sensor, for receiving motion-based gestures through motion of theuser device; and a controller, coupled to the motion sensor, configuredto control an interactive application of the user device according tothe motion-based gestures. 2-21. (canceled)
 22. The interactivecomputing system of claim 1, wherein the motion-based gestures comprisea plurality of predefined gestures.
 23. The interactive computing systemof claim 1, wherein the motion-based gestures are defined by localmotion of the user device.
 24. The interactive computing system of claim1, wherein the motion-based gestures comprise one or a combination of: aleft tilt, a right tilt, a forward tilt, a rear tilt, an inversion tilt,a circular tilt, a flip tilt, a twist, and a shake of the user device.25. The interactive computing system of claim 1, further including anaccelerometer, wherein the motion-based gestures are determined at leastin part according to data from the accelerometer.
 26. The interactivecomputing system of claim 1, further including a gyroscope, a compass ora positioning system, wherein the motion-based gestures are determinedat least in part according to data from the gyroscope, the compass orthe positioning system. 27-28. (canceled)
 29. The interactive computingsystem of claim 1, wherein the controller is configured to cause theinteractive application to purchase an item, add an item to a digitalshopping cart or wish list, place a bid on an item, complete atransaction, make a phone call, send an SMS, open another application,communicate with a further device, or call a function of anotherapplication.
 30. The interactive computing system of claim 1, furtherincluding a media player, wherein the controller is configured tocontrol the interactive application of the user device while the mediaplayer is playing media.
 31. The interactive computing system of claim30, wherein control of the interactive application is independent of theplayback of the media.
 32. The interactive computing system of claim 1,wherein the controller forms part of the user device.
 33. Theinteractive computing system of claim 1, wherein the system consists ofa handheld device.
 34. (canceled)
 35. The interactive computing systemof claim 1, wherein the controller is configured to control theinteractive application according to a set of control instructions andthe motion-based gestures, wherein the control instructions define arelationship between the motion-based gestures and the control of theinteractive application. 36-39. (canceled)
 40. An interactive methodcomprising: receiving motion-based gestures through movement of a userdevice; and controlling an interactive application of the user devicebased upon the motion-based gestures.
 41. The interactive computingsystem of claim 30, wherein the controller is configured to control theinteractive application of the user device while the media player isplaying media in full screen on the user device.
 42. The interactivecomputing system of claim 31, wherein the media comprises video.
 43. Theinteractive computing system of claim 30, wherein the motion-basedgestures include media control gestures and application controlgestures, wherein the controller is configured to control playback ofthe playing media according to the media control gestures, and controlthe interactive application based upon the application control gestures.44. The interactive computing system of claim 1, wherein the controlleris configured to selectively control the interactive application byreceiving a first motion-based gesture and determining a firstinteractive application configuration in response thereto, andsubsequently receiving a second motion-based gesture and determining asecond interactive application configuration in response thereto. 45.The interactive computing system of claims 1, wherein the control of theinteractive application is dynamically defined.
 46. The interactivecomputing system of claim 45, wherein the control of the interactiveapplication is dynamically defined according to when the motion-basedgesture was received, or a state of the system at the time the gesturewas received.
 47. The interactive computing system of claim 1, whereinthe control of the interactive application comprises a multi-dimensionalnavigation hierarchy defined using the motion-based gestures.